-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP][DiffusionDet] Diffusion models for object detection #10238
base: main
Are you sure you want to change the base?
Conversation
(inspired from frcnn modelling of transformers and ROIPooler class of detectron2)
The original model uses lists of detectron |
…tron format to bbox in torch format
Add SinusoidalPositionEmbeddings class, Complet DiffusionDetHead
hi @HichTala thanks for the PR, the model is a bit of out dated at this point I think |
Hi @yiyixuxu, |
This pull request introduces a new model,
DiffusionDet
, and its associated components. The changes include the addition of configuration, head, loss, and model files, as well as a temporary debug script to test the model.This is a first draft, nothing ready to use right now, I am creating this PR to get some feedback as it is my first time.
Key changes:
New Model Addition:
src/diffusers/models/diffusiondet/modeling_diffusiondet.py
: Added theDiffusionDet
class to implement the DiffusionDet model. This includes the model's initialization and forward pass.Configuration:
src/diffusers/models/diffusiondet/configuration_diffusiondet.py
: Added theDiffusionDetConfig
class to handle the configuration settings for the DiffusionDet model.Head Implementation:
src/diffusers/models/diffusiondet/head.py
: Added theDiffusionDetHead
class and supporting functions for the model's head, including theROIPooler
class for region of interest pooling.Loss Computation:
src/diffusers/models/diffusiondet/loss.py
: Added theDynamicCriterion
class to compute the loss for the DiffusionDet model, including the process for Hungarian assignment and supervision of matched pairs.Debug Script:
debug_diffusiondet.py
: Added a script to create and test the DiffusionDet model using the configuration and model classes.# What does this PR do?Fixes # (issue)
Before submitting
documentation guidelines, and
here are tips on formatting docstrings.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@patrickvonplaten and @ShoufaChen participated in the previous implementation trial.